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EFFECTIVE  STINGER  TRAINING  IN  RADES 


Introduction  and  Background 


Air  Defense  Training  Problems 


Realistic,  collective  training  of  Forward  Area  Air 
Defense  ( FAAD )  units  is  essential  for  combat  readiness.  However, 
the  air  defense  artillery  (ADA)  community  has  a  limited  number  of 
options  currently  available  for  pursuit  of  this  requirement. 
Training  simulators,  such  as  the  Moving  Target  Simulator  (MTS), 
do  not  allow  the  conduct  of  collective  training  and  introduce 
artifacts  not  representative  of  the  true  combat  environment. 

Field  test  exercises  are  always  resource  intensive  (costly),  are 
difficult  to  control,  and  usually  do  not  provide  realistic  threat 
targets  or  accurate  and  rapid  corrective  instructional  feedback. 

The  U.S.  Army  National  Training  Center  (NTC)  at  Fort 
Irwin,  California  employs  scaled  threat  aircraft  against  weapons 
firing  live  rounds  or  using  the  Multiple  Integrated  Laser 
Engagement  System  (MILES).  NTC  allows  greater  realism  than  do 
conventional  FTX  applications  and  provides  greater  control,  and 
more  accurate  and  rapid  feedback.  Yet  NTC  is  resource  intensive, 
heavily  scheduled,  and  provides  limited  opportunity  to  train  FAAD 
crews/teams.  In  this  regard,  NTC  cannot  be  an  effective  and 
efficient  vehicle  for  the  comprehensive,  collective  training  of 
air  defense  units. 

The  solution  to  the  problem  is  for  the  air  defense 
community  to  develop  its  own,  requirements-driven ,  collective 
training  system.  The  system  must  provide  valid,  reliable, 
realistic,  practical,  and  economic  training  with  accurate, 
rapidly-available  feedback.  The  needed  system  should  provide 
dozens  of  training  trials  each  day,  under  varying  atmospheric 
conditions,  for  most  days  of  the  year.  The  system  needed  should 
be  sufficiently  flexible  to  allow  the  training  of  single  fire 
units,  squads/sections,  and/or  a  platoon.  Finally,  the  training 
system  should  not  burden  the  air  defense  community  with 
unreasonably  high  demands  for  resources  (dollars,  equipment, 
personnel,  space,  and  time). 
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A  Potential  Solution  to  the  Training  Problems 


1 


The  Realistic  Air  Defense  Engagement  System  (RADES)  is 
a  valid  (Drewfs,  Barber,  Johnson  &  Frederickson ,  in  press)  FAAD 
testbed  located  in  the  desert  at  White  Sands  Missile  Range,  New 
Mexico.  RADES  was  designed  and  developed,  and  is  currently 
operated  and  maintained  by  Science  Applications  International 
Corporation  (SAIC)  on  behalf  of  the  Army  Research  Institute 
( ARI ) .  RADES  employs  actual  FAAD  units  and  their  weapons  in 
simulated  engagements  of  1/7  scale,  flying,  fixed-wing  (FW)  and 
1/5  scale,  "pop-up",  rotary-wing  (RW)  targets.  RADES  uses 
subscale  aircraft,  and  thus  only  requires  about  l/5th  of  the  land 
area  required  by  comparable  full-scale  field  test  exercises. 

Currently,  RADES  can  provide  dozens  of  realistic, 
collective  training  trials,  with  accurate  feedback  every  fair 
weather  day.  However,  RADES  cannot  operate  during  heavy  winds  or 
precipitation.  RADES  is  capable  of  training  one  fire  unit  at  a 
time  (to  be  upgraded  to  five).  RADES  requires  a  compliment  of 
approximately  twelve  fulltime  personnel  to  support  the  training 
of  that  fire  unit. 

RADES  provides  Short  Range  Air  Defense  (SHORAD) 
soldiers  the  opportunity  to  practice  and  to  generalize  part-task 
engagement  behaviors  in  a  representative  combat  environment. 
Criterion-guided  instructor  feedback  can  be  focused  on  the 
soldier's  and  team's  responses  during  realistic  fixed  and  rotary¬ 
wing  aircraft  scenarios.  Learning  obtained  from  the  classroom 
and  part-task  trainers  can  be  further  enhanced  using  the  whole- 
task  environment  of  RADES.  The  knowledge  and  skills  of  the 
soldier  can  be  shaped  into  meaningful  soldier  behaviors  using 
RADES.  If  RADES  successfully  "trains",  engagement  event  reaction 
times  will  be  reduced,  engagement  event  ranges  will  be  increased, 
and  target  identification  accuracy  will  be  improved.  This  study 
and  subsequent  RADES  training  applications  will  be  dedicated  to 
answering  the  question,  "Does  RADES  train?". 

In  the  current  configuration,  RADES  cannot  provide  a 
complete  solution  to  the  air  defense  collective  training  problem. 
RADES  requires  too  many  resources,  to  train  too  few  units,  during 
too  few  days  of  the  year.  Some  of  these  shortfalls  can  and  are 
being  corrected.  Before  undertaking  the  modification  of  RADES 
into  a  collective  unit  trainer,  however,  it  becomes  necessary  to 
assess  the  effectiveness  of  RADES  in  specific  types  of  training 
applications. 
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Training  Effectiveness  Issues 


First,  a  trainer  must  be  shown  to  produce  a  positive 
change  in  performance  as  a  result  of  reinforced  practice  and 
feedback.  That  is,  a  subject  team  should  perform  superior  to  its 
entry  level  criterion  measure  scores,  and  should  demonstrate  a 
systematic  learning  gain  across  a  set  of  repeated  trials  as  a 
result  of  training.  In  such  repeated  measures  testing,  the 
subject  team  acts  as  its  own  control;  responses  to  the  treatments 
are  measured  in  terms  of  deviations  about  a  point  which  measures 
the  average  performance  of  that  individual  team  (Winer,  1971; 
Kerlinger  &  Pedhazur,  1973). 

The  second  issue  is  that  the  training,  so  obtained, 
must  be  valid  (i.e.,  what  is  purported  to  be  trained  is  actually 
trained).  Are  RADES-trained  air  defenders  better  than  non-RADES- 
trained  air  defenders?  This  seemingly  obvious  question  is 
difficult  to  answer  definitively.  Validation  of  training  in 
combat  arms  ultimately  requires  simulated  warfare.  RADES  is 
currently  the  most  valid  and  realistic  Forward  Area  Air  Defense 
simulation  available.  However,  RADES  validity  as  a  trainer  may 
be  limited  to  ecological,  or  face  validity. 

The  third  issue  is  characterized  by  the  question,  "Is 
the  training  achieved  using  RADES  more  cost  effective  than 
existing  and  competing  alternatives?"  This  issue  poses  a  dilemma 
for  ARI ,  as  no  existing  trainer  or  facility  purports  to  train  all 
of  the  same  engagement  event  efficiency  and  effectiveness  factors 
that  RADES  does. 

Given  the  aforementioned  issues,  ARI  has  focused  on  the 
first  training  issue:  the  training  effect.  ARI  needs  to  know  if 
training  with  RADES  produces  a  reliable  increment  in  part-task 
and  whole- task  air  defense  engagement  performance.  The  first  of 
two  studies  planned  to  investigate  the  training  effect  issue  is 
the  current  Stinger  training  study.  The  second  study  will  be  the 
Multiple  RADES  Stinger  training  study. 
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Method 


The  present  study  investigated  the  performance  of 
Stinger  teams  during  air  defense  simulations  using  the  Realistic 
Air  Defense  Engagement  System.  Effects  of  repeated  trials# 
training,  and  feedback  on  measures  of  aircraft  engagement 
response  times#  response  ranges#  and  response  accuracy  were 
analyzed. 


The  primary  objective  of  the  test  was  to  provide 
realistic  and  representative  scenarios  to  soldiers  for  practice 
and  rehearsal  of  their  air  defense  knowledge  and  skills. 

Secondary  to  this  was  the  objective  of  measuring  improvements  in 
performance  using  a  repeated  measures  design.  In  order  to 
provide  sufficient  and  continuing  training  to  the  troops,  some 
compromises  to  the  second  objective  occurred  resulting  in  missing 
fixed-wing  data.  Therefore#  only  rotary-wing  data  were  used  in 
the  repeated  measures  comparisons.  Due  to  a  small  sample  size, 
statistical  tests  used  were  non-parametric  (sign  and  Wilcoxon 
matched  pairs  tests);  a  one-tailed  alpha  of  .1  was  designated  as 
the  criterion  for  statistical  significance  in  testing  the 
hypothesis  that  performance  in  the  posttest  would  be  superior  to 
that  of  the  pretest. 

Fourteen  scenario  treatments  (trials)  were  administered 
to  each  team#  each  followed,  by  corrective  instructional  feedback. 
Training  feedback  lessons  were  administered  by  a  platoon 
sergeant,  to  prevent  experimenter  intervention.  The  training 
scenario  treatments  were  grouped  into  four  training  experiments. 
These  training  experiments  differed  in  the  type  of  aircraft 
stimuli  used  and  the  number  of  aircraft  presented.  The  four 
experiments  have  been  described  in  Table  1.  All  ten  teams 
received  an  equivalent  form  of  each  experiment. 
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Table  1 

Repeated  Trial  Training  Experiments 


|  Experiment 

1 

2 

3 

4  1 

|  Targets 

|  SFW  | 

SRW  | 

|  SEQMRW 

SIMMRW  | 

SFW  =  Single  Fixed-Wing  Scenarios. 

SRW  =  Single  Rotary-Wing  Scenarios. 

SEQMRW  =  Sequential  Multiple  Rotary-Wing  Scenarios. 

These  served  as  pretest  and  posttest  scenarios. 

SIMMRW  =  Simultaneous  Multiple  Rotary-Wing  Scenarios. 


Each  of  the  four  training  experiments  represented  a 
combination  of  equivalent  scenarios  and  target  presentations  (see 
Table  2).  For  purposes  of  analysis,  the  double  and  triple 
simultaneous  rotary-wing  treatments  were  combined  into  a  single 
experiment,  called  Experiment  4.  Doing  this  increased  the  number 
of  cases  for  1st  and  2nd  target  engagements,  without  compromise 
to  3rd  target  engagements. 

The  fourteen  scenarios  for  each  scenario  set  are 
described  in  Table  2.  In  Table  2  the  fixed-wing  (FW)  scenario 
patterns  of  11,  12  and  1  o'clock,  refer  to  the  approach  azimuth 
of  these  target  stimuli,  relative  to  the  Stinger  weapon  position. 
In  the  case  of  rotary-wing  (RW)  experiments,  the  pattern  refers 
to  one  of  six  helicopter  target  stands  labeled  by  the  numbers  2 
to  7 .  Stand  2  is  the  leftmost  stand  and  stand  7  is  the  rightmost 
stand,  relative  to  the  central  weapon  position.  Helicopter 
target  numbering  proceeded  from  left  to  right  in  numeric  order. 
Helicopter  stands  were  approximately  equal  distance  from  the 
weapon  position,  at  a  constant  range  of  600  meters  (1/5  scale) 
equating  to  3,000  full-scale  meters. 
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Table  2 

Stinger  Training  Scenarios 


Scenario  No. 

Set  1|  Set  2 

Target /Pattern 

Set  1  |  Set  2 

Aircraft 

Presentation 

Duration 

1 

15 

llo’clk 

llo' elk 

MIG-27 

SINGLE 

N/A 

2 

16 

12o ' elk 

12o' elk 

SU-25 

SINGLE 

N/A 

3 

17 

lo ' elk 

lo ' elk 

A-7 

SINGLE 

N/A 

4 

18 

12o ' elk 

12o' elk 

A— 10 

SINGLE 

N/A 

5 

19 

7 

3 

AH— 1 

SINGLE 

20  SECS 

6 

20 

2 

7 

UH-1 

SINGLE 

20  SECS 

7 

21 

4 

4 

CH-3 

SINGLE 

20  SECS 

8 

22 

6 

2 

MI-24 

SINGLE 

20  SECS 

9 

23 

3 

6 

MI-28 

SINGLE 

20  SECS 

10 

24 

5 

5 

MI-8 

SINGLE 

20  SECS 

11 

25 

5,7 

5,3 

MI -8;  AH— 1 

SEQUENTIAL 

20  SECS 

12 

26 

4,3 

4,6 

CH-3;  MI-28 

SEQUENTIAL 

20  SECS 

13 

27 

6,3 

2,6 

MI-24;  MI-28 

SIMULTANEOUS 

40  SECS 

14 

28 

2,4,6 

7,4,2 

UH-1;  CH-3 

SIMULTANEOUS 

40  SECS 

MI-24 

NOTE:  RW  targets  2  and  3  were  at  11  o'clock; 

RW  targets  4  and  5  were  at  12  o'clock; 

RW  targets  6  and  7  were  at  1  o'clock. 


Participating  Stinger  teams  were  divided  into  two 
groups:  the  morning  group  and  the  afternoon  group.  Morning 

group  teams  received  the  first  set  of  fourteen  scenarios,  and  the 
afternoon  group  received  an  equivalent  set  of  these  fourteen 
scenarios.  The  scenario  training  sets  were  administered  to  the 
teams  at  alternating  times  of  day,  as  shown  in  Table  3. 


6 


Table  3 

Morning  and  Afternoon  Groups 


|  DAY 

MORNING 

TEAM 

AFTERNOON 

TEAM  | 

1 

SET 

1 

1 

SET  2 

2 

2 

SET 

2 

3 

SET  1 

4 

3 

SET 

1 

5 

SET  2 

6 

4 

SET 

2 

7 

SET  1 

8 

5 

SET 

1 

9 

SET  2 

10 

Fixed-wing  aircraft  (1/7  scale)  flew  at  a  constant  rate 
of  98  miles  per  hour,  (which  equates  to  about  600  knots  full- 
scale).  Fixed-wing  targets  maintained  a  constant  altitude  of 
about  50  to  75  feet,  approximating  a  full  scale  altitude  of  350 
to  525  feet.  Fixed-wing  aircraft  turned  into  the  engagement  zone 
at  an  equivalent  full-scale  range  of  18,500  meters  to  20,000 
meters  and  came  to  within  1,000  to  1,500  full-scale  meters  of  the 
weapon  position  prior  to  turning  from  the  inbound  to  the  outbound 
flight-path  segment. 

The  order  of  scenario  treatment  presentations  was 
randomized,  except  for  the  pretest  scenarios  (scenario  11  and  the 
equivalent  form  25),  which  were  always  presented  first,  and  the 
posttest  scenarios  (scenario  12  and  the  equivalent  scenario  26), 
which  were  always  presented  last.  Table  4  depicts  the  scenario 
order  of  presentation  for  the  training  treatments  for  each  team. 
It  should  be  noted  that  in  all  cases  but  the  fixed-wing 
scenarios,  (i.e.,  scenarios  1-4  and  scenarios  15-18),  the  order 
of  scenario  administration  was  maintained.  The  fixed-wing 
aircraft  were  presented  on  an  as-available  basis. 
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Table  4 

Stinger  Training  Scenario  Order 


|  SET  1  SCENARIOS  | 

ORDER 

1ST 

2ND 

3RD 

4TH 

5TH 

6TH 

7TH 

8TH 

9TH 

10TH 

11TH 

12TH 

13TH 

14TH 

TEAM 

i 

11 

5 

13 

2 

10 

8 

3 

1 

6 

4 

7 

14 

9 

12 

4 

11 

9 

4 

2 

14 

1 

3 

6 

10 

8 

13 

5 

7 

12 

5 

11 

3 

5 

1 

7 

6 

14 

9 

8 

4 

10 

2 

13 

12 

8 

11 

7 

1 

6 

9 

14 

8 

2 

3 

13 

10 

5 

4 

12 

9 

11 

4 

2 

9 

8 

7 

6 

10 

13 

3 

5 

14 

1 

12 

SET  2  SCENARIOS 


ORDER 

TEAM 

1ST 

2ND 

3RD 

4TH 

5TH 

6TH 

7TH 

8TH 

9TH 

10TH 

11TH 

12TH 

13TH 

14TH 

2 

25 

15 

17 

24 

23 

20 

16 

27 

28 

22 

21 

18 

19 

26 

3 

25 

24 

17 

22 

16 

15 

20 

19 

27 

23 

28 

18 

21 

26 

6 

25 

15 

19 

18 

23 

24 

22 

17 

16 

20 

27 

28 

21 

26 

7 

25 

17 

21 

18 

23 

19 

20 

28 

24 

22 

16 

15 

27 

26 

10 

25 

1 

20 

21 

15 

22 

27 

18 

23 

24 

1 

28 

17 

19 

16 

26 

Dependent  Measures 


The  dependent  measures  were  task  performance  measures, 
performance  effectiveness  measures,  and  summary  performance 
measures.  Engagement  task  performance  measures,  defined  in  Table 
5,  included  the  range  and  time  of  target  detection,  acquisition, 
interrogation,  identification,  engagement  command,  lock-on,  and 
fire.  Performance  effectiveness  measures  included  percent 
targets  detected,  identified,  identified  correctly,  engaged, 
destroyed,  and  handed-off  correctly.  Summary  performance 
measures  included  the  ratio  of  hostiles  destroyed  to  available 
hostiles  (percent  attrition),  ratio  of  friendlies  destroyed  to 
available  friendlies  (percent  fratricide),  and  percent  of 
hostiles  delivering  ordnance  (asset  vulnerability).  This  last 
measure  was  based  on  ordnance  delivery  time  or  range,  time  or 
range  of  weapon  fire,  and  elapsed  time  or  range  to  missile- target 
intercept.  RW  ordnance  release  time  was  designated  arbitrarily 
as  18  seconds  from  availability.  FW  ordnance  release  range  was 
designated  arbitrarily  as  4  kilometers  from  the  weapon. 

Participants 


Participants  in  the  present  study  were  ten  Stinger 
teams,  consisting  of  one  team  leader  and  one  gunner  each,  from 
the  U.S.  Army  Air  Defense  Artillery  School  (USAADASCH),  Fort 
Bliss,  Texas.  Soldiers  averaged  about  five  months  in  the 
service,  and  three  and  a  half  months  in  their  MOS.  Table  6 
provides  descriptive  statistics  in  terms  of  age  and  visual 
capabilities  measured  using  the  RADES  Vision  Testing  System. 

These  data,  along  with  performance  data,  were  used  by  personnel 
from  the  School  Brigade,  TSM-FAAD ,  Air  Defense  Board,  and  ARI  to 
balance  samples  of  soldiers  participating  in  a  subsequent  "shoot 
off"  test  of  alternative  Pedestal  Mounted  Stinger  weapon  systems. 
It  should  be  noted  that  the  Stinger  team  members  were  recent 
graduates  of  Advanced  Individual  Training,  U.S.  Army  Air  Defense 
Artillery  Training  Center,  Fort  Bliss,  Texas.  The  assignment  of 
positions  within  the  teams  was  random. 
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Table  5 

Dependent  Variables 


CODE  |  TITLE/DESCRIPTION  |  DUTY  | INTERPRETATION 


IDCOR 

Correctness  of 
Identification 

TL 

Number  of  correct 
identifications 
divided  by  number  of 
targets  identified 

RDET 

Range  of  Detection 

TL  or  G 

The  slant  range  from 
the  weapon  to  the 
target  when  the  event 
took  place;  greater 
ranges  usually 
indicate  better 
performance  for 
detection  and 
identification  but 
not  always  for  the 
other  events  (target 
can  be  inbound  or 
outbound).  Range  is 
relevant  for  fixed- 
wing  targets  only 
since  rotary-wing 
targets  simply 
popped-up  from  a 
static  position 

RID 

Range  of  Identification 

TL 

RENG 

Range  of  Command  to  Engage 

TL 

RACQ 

Range  of  Initial 
Acquisition 

—  — 

G 

RIFF 

Range  of  Interrogation 

G 

RLOCK 

Range  of  Lock-On 

G 

RFIRE 

Range  of  Weapon  Fire 

G 

TDET 

Time  of  Detection 

TL  or  G 

Based  on  seconds 
after  target 
availability  where 
availability  begins 
once  the  computer 
commands  the  target 
to  rise 

TID 

Time  of  Identification 

TL 

Time  interval  between 
Detection  and 
Identification 

TENG 

Time  of  Command  to  Engage 

TL 

Time  interval  between 
Identification  and 
Command  to  Engage/ 

Cease  Engagement 

TACQ 

Time  of  Initial 

Acquisition 

G 

Time  interval  between 
Detection  and 
Acquisition 

TIFF 

Time  of  Interrogation 

G 

Time  interval  between 
Detection  and  IFF 
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Table  5  (continued) 


TLOCK 

Time  of  Lock-On 

G 

Time  interval  between 
Acquisition  and 

Lock-on 

TFIRE 

Time  of  weapon  Fire 

G 

Time  interval  between 
Lock-on  and  Fire 

TTOT 

Total  Engagement  Time 

Both 

ime  interval  between 
Detection  and  Fire 

THAND 

Time  of  Handoff 

Both 

Time  interval  between 
Command  to  Engage  and 
Weapon  Fire 

KEY 


TL 

G 


Team  Leader 
Gunner 


Table  6 

Sample  Descriptive  Statistics  (N  =  18) 


j VARIABLE 

CODE 

MEAN 

STD  DEV 

MIN 

MAX  | 

AGE 

AGE 

19.9 

2.5 

17.0 

27.0 

NEAR  ACUITY 

NA 

7.5 

1.6 

6.0 

13.0 

CONTRAST  SENSITIVITY 

CS 

1.8 

0.4 

1.2 

2.4 

DARK  FOCUS 

DF 

1.2 

0.8 

-0.4 

2.6 

NEAR  POINT  FOCUS 

NP 

8.6 

1.9 

5.4 

11.0 

FAR  POINT  FOCUS 

FP 

-0.9 

0.8 

-3.2 

0.0 

FOCAL  RANGE 

FR 

9.3 

2.0 

5.8 

12.8 

KEY:  UNITS  OF  MEASUREMENT 

AGE:  Years 

NEAR  ACUITY:  Height/width  of  smallest  characters  resolvable  at 
20  feet  in  n/32  inches  (12/32=  20/20) 

CONTRAST  SENSITIVITY:  Average  contrast  interval  at  detection 

across  5  frequency  gratings  (min  =  1, 
max  =  5 ) 

DARK  FOCUS:  Diopters 
NEAR  POINT  FOCUS:  Diopters 
FAR  POINT  FOCUS:  Diopters 
FOCAL  RANGE:  Diopters 


NOTE:  For  vision  variables,  small  values  indicate  better 

performance  for  NA,  CS,  DF,  and  FP  and  large  values 
indicate  better  performance  for  NP  and  FR. 
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The  Stinger  Missile  System 


The  Stinger  is  a  man-portable  air  defense  (MANPAD), 
shoulder- fired ,  infrared- homing  (heat  seeking)  guided  missile 
system.  The  weapon  requires  no  control  from  the  gunner  after 
firing.  Stinger  has  an  identification  friend  or  foe  (IFF) 
subsystem  which  aids  the  team  chief  in  identifying  friendly 
aircraft.  The  Stinger  weapon  system  consists  of  four  components: 
the  weapon  round,  IFF  subsystem,  shipping  and  storage  containers, 
and  transport  harness.  The  Stinger  weapon  round  is  made  up  of  a 
missile  round  consisting  of  a  Stinger  missile  within  a  launch 
tube  mated  to  a  separate  gripstock.  A  battery/coolant  unit  ( BCU ) 
is  inserted  into  the  weapon  round  to  provide  prelaunch  power  to 
the  system.  All  of  the  components,  including  the  missile, 
separate  gripstock,  IFF  antenna,  and  BCU,  are  necessary  to 
achieve  an  operational  weapon.  The  weapon  is  60  inches  long,  and 
with  BCU  inserted,  weighs  34.7  pounds.  For  IFF  simulation 
capability,  an  IFF  interrogator  is  connected  by  cable  to  the 
weapon.  The  Stinger  tracking-head  trainer  ( THT )  which  is  a 
replica  of  the  actual  weapon,  was  used  during  the  conduct  of  this 
study.  The  THT  weighs  about  40  pounds  and  simulates  all  the 
operational  characteristics  of  the  Stinger  weapon.  The  training 
given  to  MANPAD  teams  during  this  test  was  performed  using  the 
Stinger  THT. 


Environment 


The  test  was  conducted  in  mid-September,  1986,  under 
clear  day  conditions.  The  mean  visibility  range  was  50  miles, 
and  temperatures  ranged  between  70  and  90  degrees.  Windspeed 
ranged  from  0  to  30  miles  per  hour.  The  test  environment 
consisted  of  desert  terrain,  with  no  intermediate  terrain  masking 
of  FW  or  RW  targets.  Targets  were  presented  to  the  south  of  the 
weapon  position. 
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Procedures 


Teams  were  alerted  10  seconds  prior  to  target 
availability  by  a  constant  alert  message  of  "Red-Tight".  This 
message  established  the  air  defense  warning  condition  as  "Red", 
meaning  aircraft  imminent  and  the  weapon  control  status  as 
"Tight",  meaning  aircraft  must  be  visually  identified  as 
"hostile"  prior  to  engaging.  Prior  to  the  10  second  alert 
message,  teams  were  oriented  due  north,  away  from  the  active 
search  sector,  in  a  seated  position,  which  was  in  defilade  to  the 
target  area.  Upon  issuance  of  the  "Red-Tight"  alert  message, 
teams  manned  the  weapon  position  in  preparation  for  the  target. 
After  each  trial  ended,  the  teams  were  given  a  message  which 
informed  them  that  the  alert  status  had  changed  from  "Red"  to 
"White".  Teams  returned  to  positions  facing  away  from  the  target 
area  upon  receiving  this  message.  The  weapon  was  temporarily 
stored  on  a  platform  between  trial  presentations. 

The  aircraft  interrogation  process  was  simulated  using 
an  IFF  Simulator,  modified  to  emit  only  an  "Unknown"  return. 

This  was  done  to  force  teams  to  positively  identify  each  aircraft 
as  "friend"  or  "hostile".  No  directional  (azimuth  orienting) 
cues  were  given.  All  targets  appeared  within  a  90  degree  search 
sector  that  had  to  be  visually  searched  without  optical  aiding  to 
detect  each  target.  The  visual  search  process  was  allowed  to 
vary  among  teams.  Team  chiefs  used  standard  issue  7x5 0  power 
binoculars  for  the  purpose  of  visual  aircraft  identification. 

Subject  teams  were  trained  in  and  forced  to  follow  the 
following  target  engagement  event  sequence  logic: 


1.  Both  team  members  visually  SEARCH  the 
sector  for  aircraft. 

2.  Any  team  member  DETECTS  the  aircraft  and 
announces  the  word  "Contact"  along  with  a 
clock  azimuth  denoting  its  approach. 

3.  Gunner  INTERROGATES  the  target  (now  in 
his  sight  reticle)  by  depressing  IFF 
switch  and  announces  the  result. 

4.  Team  Chief  visually  IDENTIFIES  aircraft 
and  announces  it  as  "Hostile"  or 
"Friendly" . 

5.  Gunner  ACQUIRES  the  target  in  the  sight 
reticle  by  getting  IR  tone. 
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6.  Gunner  LOCKS-ON  to  the  target  using  the 
tone  signal  generated  by  the  infrared 
seeker  head. 

7.  Team  Chief  commands  either  "ENGAGE"  or 
"CEASE  ENGAGMENT"  . 

8.  Gunner  FIRES  or  BREAKS-OFF  from  the 
current  target  engagement. 


The  detection/  identification  and  command  to 
engage/cease  engagement  events  were  time-coded  and  logged  via 
manual  keystroke  entries  made  by  an  observer.  The  interrogate, 
acquire,  lock-on  and  fire  events  were  time-coded  and  logged  via 
direct  circuit  taps  on  the  weapon  system.  Each  event  was 
correlated  to  the  range  of  the  aircraft  and  the  time  elapsed 
after  target  availability. 


Training  Feedback 


Following  each  training  trial,  a  feedback  screen  was 
presented  to  the  platoon  sergeant  acting  as  the  training 
instructor.  The  time  from  end-of-trial  to  presentation  of 
graphic  feedback  at  the  weapon  averaged  three  minutes.  An  ARI 
research  scientist  interpreted  the  feedback  screen  for  the 
instructor,  who  in  turn,  gave  a  corrective  critique  to  the 
Stinger  team  chief  and  gunner. 

An  example  feedback  screen  for  a  fixed-wing  scenario  is 
presented  in  Figure  1.  Figure  2  provides  an  example  rotary-wing 
feedback  screen  display.  These  figures  were  generated  from  dummy 
data,  but  represent  the  kind  of  information  given  as  feedback. 
Information  was  presented  in  both  tabular  and  graphic  form.  The 
fixed-wing  feedback  display  provided,  for  each  target,  event 
times,  ranges,  and  target  model  identity.  Symbols  for  each  of 
the  sequential  events  were  placed  along  the  flight  path  to  enable 
team  members  to  judge  the  relative  range  of  the  target  at  event 
occurrence.  Rotary-wing  displays  differed  in  one  respect  from 
the  fixed-wing  feedback  displays.  Since  rotary-wing  targets  were 
presented  at  a  constant  range,  the  range  values  on  the  display 
did  not  vary. 


15 


FIGURE  1 

FIXED  WING  SCENARIO  FEEDBACK  SCREEN 
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FIGURE  2 

ROTARY  WING  SCENARIO  FEEDBACK  SCREEN 
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Results 


Three  categories  of  data  analysis  activities  are 
reported:  descriptive  statistics,  correlational  statistics,  and 

comparison  statistics.  The  descriptive  statistics  provide  an 
indication  of  the  average  performances  across  all  teams  for  each 
experiment.  The  correlations  provide  information  as  to  the 
relationships  between  team  attributes  and  performance.  The 
comparison  statistics  were  employed  to  test  the  hypothesis  of 
improved  posttest  performance  in  response  to  previously  presented 
scenarios.  The  results  of  the  comparison  between  pretest  and 
posttest  scenario  performance  will  be  reported  first. 


Comparison  Tests 


The  pretest  scenarios  for  the  two  scenario  sets  were  11 
and  25  (see  Table  4).  The  posttest  scenarios  for  the  two  sets 
were  12  and  26.  These  scenarios  represented  double,  sequential, 
rotary-wing  target  presentations.  An  attempt  was  made  to  insure 
that,  across  sets,  the  pretest  and  posttest  scenarios  were 
equivalent  for  clock  azimuth,  target  intent,  and  target  size. 

The  pretest  targets  were  the  large  Soviet  Mi-8s  at  12  o'clock  and 
the  small  U.S.  AH-1  at  either  1  o'clock  or  11  o'clock 
respectively.  The  posttest  targets  were  the  large  U.S.  CH-3  at 
12  o'clock,  and  the  small  Soviet  Mi-28  at  either  11  o'clock  or  1 
o'clock,  respectively. 

There  was  a  3  to  5  second  delay  between  "target 
availability"  (when  the  computer  commanded  the  helicopter  to 
rise)  and  "line-of-sight "  (when  the  helicopter  had  risen  far 
enough  to  be  seen  from  the  weapon  position).  Variability  in  this 
delay  resulted  from  variability  in  terrain,  target  system,  and 
atmospheric  conditions. 

The  hypothesis  being  tested  was  that  the  duration  of 
timed  events  subsequent  to  availability  would  be  shorter  for  the 
posttest  treatments  due  to  an  improvement  in  performance 
attributable  to  the  training  received.  The  expected  trend  was 
that  performance  would  improve  after  training,  so  a  one-tailed 
alpha  was  chosen  and  the  significance  level  was  set  at  .10. 

Since  aircraft  intent  (friend,  foe)  differed  in  the 
pre-post  treatments,  only  events  unrelated  to  weapon  engagement 
could  be  meaningfully  compared.  Target  size  and  order 
differences  also  prohibited  comparison  of  engagement  (gunner) 
actions.  Therefore,  effects  for  the  events  of  lock-on  and  fire 
will  not  be  discussed. 
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Pre-  and  post- treatment  responses  were  compared  to 
determine  if  performance  differed.  The  sign  (binomial)  test  was 
used  to  make  these  comparisons.  Table  7  presents  the  results  of 
these  comparisons. 


Table  7 

Pre-  and  Post-Treatment  Comparisons  of  Rotary-Wing  Event  Times 


PRE  |  POST  |  SIGN  TEST 


|  VARIABLE 

TARG 

MEAN 

SD 

N 

MEAN 

SD 

N 

SIGNIFICANCE | 

TDET 

1st 

15.2 

1.7 

9 

10.5 

2.1 

7 

7,0 

p<.01 

TDET 

2nd 

45.4 

5.3 

8 

40.0 

3.8 

7 

6,0 

p<  .05 

TID 

BOTH 

9.4 

3.1 

9 

4.9 

0.8 

7 

7,0 

p<  .01 

TACQ 

BOTH 

3.8 

2.1 

8 

2.9 

1.2 

7 

5,1 

p=.ll 

TIFF 

BOTH 

8.6 

7.4 

7 

3.4 

3.0 

7 

4,1 

NS 

TENG 

BOTH 

3.6 

5.6 

6 

1.0 

0.0 

8 

2,0 

NS 

TLOCK 

BOTH 

9.9 

4.4 

9 

6.3 

2.6 

7 

5,1 

p=.ll 

TFIRE 

BOTH 

2.2 

1.1 

9 

1.8 

1.2 

7 

4,1 

NS 

TTOT 

BOTH 

14.3 

4.8 

9 

9.9 

2.3 

7 

6,1 

p<  .10 

THAND 

BOTH 

5.0 

1.7 

8 

4.5 

1.6 

7 

3,3 

NS 

IDCOR 

BOTH 

.81 

.12 

9 

1.0 

.00 

7 

4,0 

p<  .10 

NOTE: 


NS  ■  not  significant 
BOTH  =  average  response  across  both 
targets  in  the  scenario 
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It  is  apparent  from  Table  7  that  the  teams  did 
significantly  better  during  the  posttest  than  they  did  during  the 
pretest.  Targets  were  detected,  acquired  and  identified  earlier; 
an  improvement  of  about  5  to  7  seconds  was  observed  during  the 
posttest.  Almost  a  50%  improvement  was  found  for  the 
identification  event  alone.  These  improvements  represent  the 
kind  of  performance  increment  desired  by  the  ADA  training 
community.  When  exposed  to  multiple  aircraft,  it  is  important 
that  the  fire  units  detect  and  identify  each  aircraft  rapidly  so 
that  the  hostiles  can  be  picked  out  and  engaged  according  to  the 
greatest  threat. 

Table  8  presents  a  comparison  of  the  pre-post 
treatments  with  respect  to  summary  performance  measures. 
Identification  accuracy  showed  improvement  during  the  posttest. 
During  the  pretest,  40%  of  the  friendly  targets  were  identified 
as  hostile;  none  of  the  hostile  targets  were  misidentif ied. 

During  the  posttest,  all  targets,  friendly  and  hostile,  were 
identified  correctly.  During  the  posttest,  no  friends  were 
engaged,  as  opposed  to  the  pretest  where  40%  of  the  friends  were 
engaged.  These  pre-post  differences  in  friendly  identifications 
and  friendly  engagements  were  both  significant  (Cochran’s  Q=4.0; 
df=l;  1-tailed  p<.05).  It  is  interesting  to  note  that  the 
shorter  posttest  engagement  event  times  were  insufficient  to 
affect  a  change  from  pretest  to  posttest  in  the  percentage  of 
hostiles  delivering  ordnance. 


Table  8 

Performance  Effectiveness  Comparisons 


PRE  |  POST 


|  VARIABLE 

% 

N 

% 

N  | 

Targets  Detected 

100 

20/20 

100 

16/16 

Targets  Identified 

100 

20/20 

100 

16/16 

Correct  Identifications 

80 

16/20 

100 

16/16 

Incorrect  Identifications 

20 

4/20 

0 

0/16 

Hostiles  Correctly  Identified 

100 

10/10 

100 

8/8 

Friends  Correctly  Identified 

60 

6/10 

100 

8/8 

Hostiles  Engaged 

100 

10/10 

100 

8/8 

Friends  Engaged 

40 

4/10 

0 

0/8 

Engaged  Aircraft  Destroyed 

93 

13/14 

100 

8/8 

Hostile  Aircraft  Destroyed 

90 

9/10 

100 

8/8 

Friendly  Aircraft  Destroyed 

40 

4/10 

0 

0/8 

Correct  Hand-offs 

100 

20/20 

100 

16/16 

Hostiles  Delivering  Ordnance 

100 

10/10 

100 

8/8 
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A  final  pre-post  comparison  test  was  conducted  to 
determine  if  team  performance  improved  after  each  subsequent 
target  presentation.  If  a  trend  existed  in  the  data 
demonstrating  a  positive  gain  as  the  training  progressed,  then  it 
could  be  inferred  that  learning  had  taken  place.  Data  from 
single  presentations  of  rotary-wing  targets  were  reviewed  since 
most  of  the  scenarios  began  with  a  single  RW  target  (e.g., 
scenarios  5,  6,  7,  8,  9,  10,  11,  12,  19,  20,  21,  22,  23,  24,  25, 
26).  The  size  of  the  aircraft  was  used  as  a  control;  large 
models  (CH3,  MI8,  MI24,  MI28)  were  reviewed  separately  from  the 
small  models  (UHl,  AHl). 

A  sign  test  was  performed  where  an  improvement  in 
reaction  time  over  a  preceding  trial  would  receive  a  (+)  and  a 
decrement  in  performance  on  a  subsequent  trial  would  receive  a 
(-).  Therefore,  each  time  a  team  was  exposed  to  a  single  RW 
target  of  the  same  size  (large  or  small),  their  responses  were 
recorded  in  terms  of  being  better  or  worse  than  before  (ties  were 
excluded).  The  results  for  each  team  for  each  of  the  two  RW 
target  sizes  were  pooled  at  the  end  to  obtain  a  tally  of  pluses 
and  minuses.  Tallies  resulting  from  the  small  RW  trial 
comparisons  were  so  low  ( n=7 )  that  they  were  pooled  with  tallies 
from  the  large  RW  trial  comparisons  (n=38)  resulting  in  a  total 
number  of  45  possible  comparisons.  A  one-tailed  test  was 
performed  on  the  resulting  tally,  the  alternative  hypothesis 
being  that  a  positive  improvement  trend  would  emerge.  The  event 
times  for  RW  detection  and  identification  were  analyzed.  The 
following  formula  (Conover,  1980)  was  used  to  obtain  the  critical 
region . 


T  >  np  +  Wr*\v/np(  1-p) 

(n  =  sample  size;  p  =  .5;  1-p  =  .5;  Wr  =  1.645  for  alpha  =  .05) 


The  results  of  this  analysis  are  summarized  in  Table  9. 
There  was  a  trend  in  the  single  RW  data  indicating  that 
performance  improved  overall  as  the  training  progressed.  This 
was  especially  true  for  the  reduction  in  identification  time 
after  target  detection.  The  improvement  trend  in  performance  on 
the  large  RW  scenarios  was  much  more  profound  than  for  the  small 
RW  scenarios.  These  findings  add  further  support  to  the 
assertion  that  RADES  can  be  used  to  provide  effective  training. 
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Table  9 

Results  of  Improvement  Trend  Tests 


|  VARIABLE 

1  +  1 

1 

|  1-TAILED  PROBABILITY  | 

TDET 

27 

17 

.07 

TID 

28 

15 

.03 

The  results  reported  above  relate  only  to  RW  engagement 
performance.  More  research  is  required  to  determine  if  any 
performance  gain  due  to  RADES  training  can  be  found  for  FW 
engagements.  Later  training  research  using  RADES  will  address 
this  and  other  relevant  issues.  However,  this  pilot  program  has 
provided  insights  in  determining  what  future  test  requirements 
and  issues  should  be  addressed. 
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Descriptive  and  Summary  Statistics 


Descriptive  statistics  were  compiled  across  all  teams, 
according  to  training  treatment,  for  each  target.  Summary 
performance  measures  for  each  treatment  (experiment)  and  target 
are  provided  in  Tables  10  to  13.  Descriptive  statistics  are 
given  in  Tables  14  to  17.  These  tables  can  be  found  at  the  end 
of  this  section. 

Table  10  presents  summary  statistics  for  the  Single 
Fixed-Wing  test  (Experiment  1).  Teams  detected  and  identified 
all  FW  target  aircraft.  However,  almost  a  third  of  the  aircraft 
were  incorrectly  identified,  with  the  majority  of  the  incorrect 
identifications  being  misidentif ications  of  friendly  aircraft. 
Thus,  hostile  fixed-wing  aircraft  (i.e.,  MIG-27s  and  SU-25s)  were 
more  often  correctly  identified  than  the  friendly  aircraft  (i.e., 
A— 7 s  and  A-10s).  In  fact,  identification  accuracy  for  friends 
was  slightly  above  chance  performance.  Almost  all  aircraft 
perceived  to  be  hostile  were  fired  upon,  resulting  in  100% 
engaged  aircraft  destroyed.  Two  of  the  hostiles  that  were 
correctly  identified  were  not  engaged,  possibly  because  of 
incorrect  handoffs,  as  handoff  accuracy  was  72%.  Friendly 
fratricide  was  high  at  44%.  Seventy-eight  percent  of  the 
hostiles  were  prevented  from  delivering  ordnance. 

Table  11  provides  summary  statistics  for  the  Single 
Rotary-Wing  test  (Experiment  2).  Again,  as  was  the  case  for 
fixed-wing  aircraft,  the  teams  detected  and  identified  100 
percent  of  the  targets.  Friendly  RW  targets  were  correctly 
identified  more  often  than  friendly  FW  aircraft.  Only  12  percent 
of  the  friendly  rotary-wing  targets  were  misidentif ied ,  and  only 
3  percent  of  the  hostiles  were  incorrectly  identified.  Again, 
hostile  aircraft  were  more  often  correctly  identified  than 
friendly  aircraft.  Stinger  teams  were  better  at  RW  scenarios 
than  FW  scenarios  in  handoffs  and  identification  accuracy,  but 
were  virtually  ineffective  in  preventing  hostile  ordnance  release 
during  the  RW  trials. 

Table  12  provides  summary  statistics  for  the  Double 
Sequential  Rotary-Wing  test  (Experiment  3).  All  first  targets 
were  detected  and  identified,  with  all  identifications  correct 
for  both  hostile  and  friendly  aircraft.  This  level  of  accuracy 
was  not  sustained  for  subsequent  targets,  where  identification 
performance  was  reduced  by  22%.  This  resulted  in  a  40%  friendly 
fratricide  rate.  All  second  target  misidentif ications  occurred 
during  the  pretest  scenarios,  as  posttest  identification  accuracy 
was  100%.  Handoff  accuracy  was  100%  for  both  targets.  All 
hostile  targets  would  have  delivered  ordnance  prior  to  their 
being  destroyed. 
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Table  13  presents  summary  statistics  for  the 
Simultaneous  Double  and  Triple  Rotary-Wing  test  (Experiment  4). 
When  all  of  the  RW  aircraft  were  presented  at  the  same  time,  the 
Stinger  teams  obviously  did  not  correctly  identify  the  first 
target  entertained  with  100  percent  accuracy.  Literally  one-half 
(overall)  of  the  friendly  RW  aircraft  in  the  multiple, 
simultaneous  aircraft  presentations  were  incorrectly  identified. 
Only  15  percent  (overall)  of  the  hostile  aircraft  were 
misidentif ied . 

Friendly  rotary-wing  identification  was  about  the  same 
in  second  target  engagements  as  it  was  in  first  target 
engagements.  However,  on  second  targets,  hostiles  were  correctly 
identified  100%  of  the  time  instead  of  88%  of  the  time.  Thus,  as 
friendly  correct  identifications  stabilized,  hostile  correct 
identifications  appeared  to  improve  in  second  aircraft 
engagements.  Interestingly,  however,  hostile  target  correct 
identifications  declined  in  third  target  engagements;  Stinger 
teams  dropped  43%  from  second  to  third  target  engagements.  Time 
management  appeared  to  suffer,  with  the  least  time  given  to 
identification  for  the  third  target. 

Table  14  presents  engagement  event  range  descriptive 
statistics  for  Experiment  1,  the  Single  Fixed-Wing  test. 
Engagement  event  target  ranges  are  in  fullscale  meters.  As  shown 
in  this  table,  targets  were  frequently  fired  at  beyond  the 
effective  range  of  the  Stinger  weapon  system.  At  maximum  range, 
several  seconds  will  be  consumed  by  the  round  flight  to  aircraft 
intercept.  Thus,  some  of  the  ingressing  aircraft  would  have  been 
effectively  destroyed,  given  adequate  lock-on  and  superelevation 
of  the  weapon  at  time  of  fire;  the  aircraft  would  have  flown 
within  range  once  the  intercept  point  was  reached.  Table  14  also 
provides  the  engagement  event  ranges  for  hostile  and  friendly 
fixed-wing  targets. 

During  Experiment  1,  the  mean  period  from  detection  to 
fire  was  30.2  seconds.  This  is  consistent  with  prior  RADES  test 
results,  which  place  this  same  average  time  interval  at  30  to  40 
seconds.  Again,  as  in  prior  RADES  experiments,  the  time  interval 
from  target  detection  to  target  identification  runs  over  half 
this  engagement  period  (detection  to  fire)  at  19.6  seconds. 

Thus,  the  team  used  about  one  third  of  this  period  for  finalizing 
the  engagement  process.  The  time  between  positive  identification 
(command  to  engage)  and  fire  is  largely  controlled  by  aircraft 
range,  since  the  gunner  must  wait  to  fire  until  the  aircraft  is 
sufficiently  close,  to  avoid  a  miss. 

Table  15  presents  engagement  efficiency  descriptive 
statistics  for  Experiment  2,  the  Single  Rotary-Wing  test.  The 
mean  period  from  detection  to  fire  was  12  seconds.  This  is 
consistent  with  prior  RADES  test  results,  which  place  this  time 
interval  at  10  to  16  seconds.  Also,  as  in  prior  RADES 
experiments,  the  time  interval  from  target  detection  to  target 
identification  runs  over  half  the  period  between  detection  and 
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fire  at  7.4  seconds.  Again,  the  team  used  about  a  third  of  this 
total  period  for  finalizing  the  engagement  process.  The  fact 
that  interrogation  times  are  late  and  there  are  many  missing 
cases  is  indicative  of  the  inexperience  of  these  subjects  as 
compared  to  previous  RADES  studies.  This  finding  is  replicated 
in  the  tables  which  follow. 

Table  16  presents  engagement  efficiency  descriptive 
statistics  for  Experiment  3,  the  Sequential  Double  Rotary-Wing 
test.  A  mean  period  from  detection  to  fire  of  15.3  seconds  was 
found  for  the  first  target.  This  value  is  also  consistent  with 
prior  RADES  tests,  as  is  the  time  interval  from  target  detection 
to  target  identification  (7.5  seconds).  Reaction  time  intervals 
for  the  second  target  were  somewhat  shorter. 

Table  17  presents  engagement  efficiency  descriptive 
statistics  for  Experiment  4,  the  Simultaneous  Double  and  Triple 
Rotary-Wing  tests.  For  target  one,,  the  mean  period  from 
detection  to  fire  was  14  seconds.  The  interval  between  detection 
and  identification  was  5.6  seconds.  These  results  again  reflect 
the  overall  consistency  of  responses  to  RADES  targets.  Responses 
to  the  second  target  were  comparable,  but  third  target  responses 
were  faster.  It  can  be  seen,  however,  that  only  about  half  of 
the  teams  were  able  to  engage  the  third  target,  indicating  a 
quickness  in  responding  by  the  leaders  and  gunners  for  those 
teams.  Further,  the  fact  that  mean  times  for  acquisition  were 
smaller  than  those  for  detection  for  second  and  third  targets  is 
indicative  of  the  greater  speed  of  some  of  the  teams  in 
completing  engagements  and  addressing  new  targets.  Some  gunners 
were  able  to  obtain  acquisition  shortly  after  detection,  thereby 
enabling  rapid  lock-on  and  fire  if  commanded  to  engage. 

In  summary,  of  the  total  engagement  period 
(availability  through  fire),  the  detection  event  takes  about  half 
of  the  time.  The  engagement  process  subsequent  to  aircraft 
detection  was  primarily  taken  up  by  the  identification  task. 

About  2/3  of  the  elapsed  time  between  detection  and  fire  involved 
identification.  The  last  1/3  of  this  time  interval  was  taken  by 
gunner  response  actions  to  the  team  leader's  command  to  engage. 
Air  Defense  training  typically  concentrates  on  these  critical 
aspects  of  the  engagement  sequence:  aircraft  detection, 
identification,  and  weapon  engagement. 

Some  general  trends  emerged  as  a  result  of  this  study. 
The  first  is  the  apparent  tendency  for  these  Stinger  teams  to 
have  a  hostile  aircraft  orientation.  Foes  were  identified 
correctly  more  consistently,  whereas  friends  were  frequently 
identified  as  foes.  This  finding  is  supportive  of  earlier  RADES 
results  demonstrating  that  novices  tend  to  possess  a  "hostile 
mind  set."  Thus,  when  in  doubt,  the  tendency  is  to  fire. 

Another  trend  is  the  apparent  increase  in  speed  of  performance  as 
the  number  of  targets  increases.  For  single  RW  trials,  and  for 
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first  targets,  teams  seemed  to  take  their  time,  whereas  second 
and  third  targets  produced  much  shorter  reaction  times.  This 
effect  may  be  related  to  team  arousal  level. 


Table  10 

Single  Fixed-Wing  —  Experiment  1  Summary  Statistics 


|  VARIABLE  | 

%  1 

N  | 

Targets  Detected 

100 

18/18 

Targets  Identified 

100 

18/18 

Correct  Identifications 

72 

13/18 

Incorrect 

28 

5/18 

Hostiles  Correctly  Identified 

89 

8/9 

Friendlies  Correctly  Identified 

56 

5/9 

Hostiles  Engaged 

67 

6/9 

Friends  Engaged 

44 

4/9 

Engaged  Aircraft  Destroyed 

100 

10/10 

Hostile  Aircraft  Destroyed 

67 

6/9 

Friendly  Aircraft  Destroyed 

44 

4/9 

Correct  Hand-offs 

72 

13/18 

Hostiles  Delivering  Ordnance 

22 

2/9 

26 


Table  11 

Single  Rotary-Wing  —  Experiment  2  Summary  Statistics 


|  VARIABLE  | 

% 

N  | 

Targets  Detected 

100 

53/53 

Targets  Identified 

100 

53/53 

Correct  Identifications 

92 

49/53 

Incorrect 

8 

4/53 

Hostiles  Correctly  Identified 

96 

27/28 

Friendlies  Correctly  Identified 

88 

22/25 

Hostiles  Engaged 

96 

27/28 

Friends  Engaged 

8 

2/25 

Engaged  Aircraft  Destroyed 

93 

27/29 

Hostile  Aircraft  Destroyed 

89 

25/28 

Friendly  Aircraft  Destroyed 

8 

2/25 

Correct  Hand-offs 

100 

53/53 

Hostiles  Delivering  Ordnance 

96 

27/28 

Table  12 

Sequential  Double  Rotary-Wing  —  Experiment  3  Summary  Statistics 


Target  1 

Target  2  j 

VARIABLE 

% 

N  1 

% 

N  | 

Targets  Detected 

100 

18/18 

100 

18/18 

Targets  Identified 

100 

18/18 

100 

18/18 

Correct  Identifications 

100 

18/18 

78 

14/18 

Incorrect 

0 

0/18 

22 

4/18 

Hostiles  Correctly  Identified 

100 

10/10 

100 

8/8 

Friendlies  Correctly  Identified 

100 

8/8 

60 

6/10 

Hostiles  Engaged 

100 

10/10 

100 

8/8 

Friends  Engaged 

0 

0/8 

40 

4/10 

Engaged  Aircraft  Destroyed 

90 

9/10 

100 

12/12 

Hostile  Aircraft  Destroyed 

90 

9/10 

100 

8/8 

Friendly  Aircraft  Destroyed 

0 

0/8 

40 

4/10 

Correct  Hand-offs 

100 

18/18 

100 

18/18 

Hostiles  Delivering  Ordnance 

100 

18/18 

100 

8/8 

27 


Table  13 

Simultaneous  Double  and  Triple  Rotary-Wing  —  Experiment  4 
Summary  Statistics 


VARIABLE 

Target  1 

%  |  N 

Target  2 

%  |  N  | 

Target  3 

%  |  N 

Targets  Detected 

100 

16/16 

100 

16/16 

100 

8/8 

Targets  Identified 

100 

16/16 

100 

16/16 

100 

8/8 

Correct  Identifications 

69 

11/16 

75 

12/16 

62 

5/8 

Incorrect 

31 

5/16 

25 

4/16 

38 

3/8 

Hostiles  Correctly  Ident. 

88 

7/8 

100 

9/9 

57 

4/7 

Friendlies  Correctly  Ident. 

50 

4/8 

43 

3/7 

100 

1/1 

Hostiles  Engaged 

75 

6/8 

100 

9/9 

57 

4/7 

Friends  Engaged 

38 

3/8 

14 

1/7 

0 

0/1 

Engaged  Aircraft  Destroyed 

78 

7/9 

100 

10/10 

100 

4/4 

Hostile  Aircraft  Destroyed 

63 

5/8 

100 

9/9 

57 

4/7 

Friendly  Aircraft  Destroyed 

38 

3/8 

14 

1/7 

0 

0/1 

Correct  Hand-offs 

87 

14/16 

100 

16/16 

100 

7/7 

Hostiles  Delivering  Ordnance 

100 

8/8 

100 

9/9 

100 

7/7 

Table  14 

Single  Fixed-Wing  Event  Ranges  (Meters)  — 
Experiment  1  Descriptive  Statistics 


j  Event 

N 

Mean 

Std  Dev 

Range 

Min 

Max  | 

RDET 

18 

12974 

2524 

7681 

8440 

16121 

RIFF 

13 

7860 

3909 

12950 

1594 

14544 

RID 

18 

8065 

3220 

12355 

1981 

14336 

RACQ 

17 

9093 

3196 

10957 

4414 

15371 

RENG 

11 

8538 

2505 

7155 

5123 

12278 

RLOCK 

10 

6047 

2734 

8975 

1856 

10831 

RFIRE 

10 

5635 

3087 

8975 

1856 

10831 

Hostile  Targets  Friendly  Targets 


|  Event 

N 

Mean 

Std  Dev 

N 

Mean 

Std  Dev| 

RDET 

9 

13643 

2945 

9 

12305 

1963 

RIFF 

6 

6988 

4903 

7 

8607 

3018 

RID 

9 

8802 

2385 

8 

7235 

3964 

RACQ 

9 

9845 

3221 

8 

8246 

3154 

RENG 

8 

8988 

2483 

2 

6738 

2284 

RLOCK 

6 

6412 

2732 

4 

5501 

3055 

RFIRE 

6 

5835 

3276 

4 

5336 

3241 

28 


Table  15 

Single  Rotary-Wing  Event  Tiroes  (Seconds)  — 
Experiment  2  Descriptive  Statistics 


|  Event 

N 

Mean 

Std  Dev 

Variance 

Range 

Min 

Max  | 

TDET 

53 

13.62 

3.25 

10.59 

13.0 

8.0 

21.0 

TID 

53 

7.38 

3.51 

12.35 

16.0 

2.0 

18.0 

TACQ 

39 

3.30 

1.44 

2.06 

5.0 

1.0 

6.0 

TIFF 

36 

8.17 

9.87 

97.46 

47.0 

1.0 

48.0 

TENG 

21 

1.90 

3.92 

15.39 

18.0 

1.0 

19.0 

TLOCK 

27 

7.33 

3.92 

15.38 

20.0 

3.0 

23.0 

TFIRE 

28 

2.00 

1.90 

3.63 

10.0 

1.0 

11.0 

TTOT 

29 

12.00 

4.77 

22.79 

24.0 

6.0 

30.0 

THAND 

28 

4.86 

3.61 

13.02 

14.0 

1.0 

15.0 

Hostile  Targets  |  Friendly  Targets 


Event 

N 

Mean 

Std  Dev 

N 

Mean 

Std  Dev 

TDET 

28 

14.61 

3.51 

25 

12.52 

2.58 

TID 

28 

5.68 

1.79 

25 

9.28 

4.01 

TACQ 

19 

3.11 

1.59 

20 

3.50 

1.28 

TIFF 

18 

6.89 

7.66 

18 

9.44 

11.77 

TENG 

15 

2.13 

3.64 

2 

1.00 

0.00 

TLOCK 

24 

7.29 

4.02 

3 

7.67 

3.79 

TFIRE 

26 

2.04 

1.97 

2 

1.50 

0.71 

TTOT 

27 

12.04 

4.93 

2 

11.50 

2.12 

THAND 

27 

4.96 

3.63 

1 

2.00 

0.00 

29 


Table  16 

Sequential  Double  Rotary-Wing  Event  Times  (Seconds)  — 
Experiment  3  Descriptive  Statistics 


Target  1 


| Event 

N 

Mean 

Std  Dev 

Variance 

Range 

Min 

Max  | 

TDET 

18 

12.67 

3.01 

9.06 

10.0 

8.0 

18.0 

TID 

18 

7.50 

4.03 

16.26 

16.0 

3.0 

19.0 

TACQ 

14 

3.64 

1.91 

3.63 

6.0 

1.0 

7.0 

TIFF 

15 

5.40 

6.50 

41.97 

21.0 

1.0 

22.0 

TENG 

7 

3.43 

5.22 

27.29 

14.0 

1.0 

15.0 

TLOCK 

8 

9.25 

5.65 

31.93 

18.0 

2.0 

20.0 

TFIRE 

9 

2.11 

0.78 

0.61 

2.0 

1.0 

3.0 

TTOT 

10 

15.30 

5.08 

25.79 

16.0 

9.0 

25.0 

THAND 

9 

5.56 

3.43 

11.78 

12.0 

2.0 

14.0 

Target  2 


1  Event 

N 

Mean 

Std  Dev 

Variance 

Range 

Min 

Max 

TDET 

17 

42.47 

5.56 

30.89 

20.0 

34.0 

54.0 

TID 

17 

6.88 

3.45 

12.23 

11.0 

2.0 

13.0 

TACQ 

9 

3.22 

2.39 

5.69 

7.0 

1.0 

8.0 

TIFF 

11 

7.09 

6.62 

43.89 

17.0 

1.0 

18.0 

TENG 

10 

1.00 

0.00 

0.00 

0.0 

1.0 

1.0 

TLOCK 

9 

7.33 

3.24 

10.50 

9.0 

3.0 

12.0 

TFIRE 

10 

2.10 

1.73 

2.99 

5.0 

1.0 

6.0 

TTOT 

12 

10.33 

4.16 

17.33 

16.0 

4.0 

20.0 

THAND 

10 

4.60 

1.78 

3.16 

5.0 

2.0 

7.0 

30 


Table  17 

Simultaneous  Double  &  Triple  Rotary-Wing  Event  Times  (Seconds) 
—  Experiment  4  Descriptive  Statistics 


Target  1 


|  Event 

N  | 

Mean 

Std  Dev 

Variance | 

Range 

Min  | 

Max  | 

TDET 

16 

11.31 

2.30 

5.30 

9.0 

9.0 

18.0 

TID 

16 

5.56 

3.93 

15.46 

17.0 

2.0 

19.0 

TACQ 

11 

2.82 

1.60 

2.56 

5.0 

1.0 

6.0 

TIFF 

9 

3.00 

1.94 

3.75 

6.0 

1.0 

7.0 

TENG 

9 

1.11 

0.33 

0.11 

1.0 

1.0 

2.0 

TLOCK 

9 

8.44 

5.81 

33.78 

15.0 

2.0 

17.0 

TFIRE 

8 

2.50 

1.64 

2.86 

5.0 

1.0 

6.0 

TTOT 

9 

14.00 

5.34 

28.50 

17.0 

8.0 

25.0 

THAND 

7 

4.86 

1.07 

1.14 

3.0 

4.0 

7.0 

Target  2 

|  Event 

N 

Mean 

Std  Dev 

Variance 

Range 

Min 

Max  | 

TDET 

16 

23.87 

6.80 

46.25 

30.0 

12.0 

42.0 

TID 

16 

7.50 

4.34 

18.80 

18.0 

3.0 

21.0 

TACQ 

1 

3.00 

0.00 

0*00 

0.0 

3.0 

3.0 

TIFF 

8 

8.37 

9.93 

98.55 

28.0 

1.0 

29.0 

TENG 

7 

1.00 

0.00 

0.00 

0.0 

1.0 

1.0 

TLOCK 

8 

15.37 

7.40 

54.84 

23.0 

2.0 

25.0 

TFIRE 

8 

2.62 

3.42 

11.70 

10.0 

1.0 

11.0 

TTOT 

8 

14.00 

7.67 

58.86 

23.0 

1.0 

24.0 

THAND 

7 

7.57 

6.38 

40.62 

16.0 

1.0 

17.0 

Target  3 

| Event 

N 

Mean 

Std  Dev 

Variance 

Range 

|  Min 

Max  | 

TDET 

8 

34.50 

8.33 

64.43 

25.0 

25.0 

50.0 

TID 

8 

5.25 

3.41 

11.64 

9.0 

1.0 

10.0 

TACQ 

1 

1.00 

0.00 

0.00 

0.0 

1.0 

1.0 

TIFF 

3 

5.33 

4.51 

20.33 

9.0 

1.0 

10.0 

TENG 

2 

1.50 

0.71 

0.50 

1.0 

1.0 

2.0 

TLOCK 

4 

6.50 

2.38 

5.67 

5.0 

5.0 

10.0 

TFIRE 

4 

1.75 

0.96 

0.92 

2.0 

1.0 

3.0 

TTOT 

4 

7.50 

3.32 

11.00 

8.0 

4.0 

12.0 

THAND 

3 

6.00 

3.46 

12.00 

6.0 

4.0 

10.0 

Correlational  Analyses 


Table  18  provides  those  correlations  between  subject 
attributes  and  dependent  variables  which  exceeded  the  .10  level 
of  significance.  Spearman  ( non-parametric )  correlation 
coefficients  were  computed  between  average  team  responses  for  FW 
and  RW  (first  targets)  and  the  team  attributes.  It  was  assumed 
that  those  variables  controlled  by  the  team  leader  (such  as 
command  to  engage)  would  be  irrelevant  for  the  gunner,  and  those 
variables  controlled  by  the  gunner  (such  as  lock-on)  would  be 
irrelevant  for  the  team  leader.  Only  relevant  relationships  are 
presented  in  Table  18. 

Although  a  few  of  the  correlations  are  conflicting,  one 
general  conclusion  that  can  be  drawn  from  Table  18  is  that  the 
vision  of  the  team  members  played  a  substantial  role  in 
accounting  for  engagement  performance.  Another  general 
observation  is  that  the  gunner's  visual  capabilities  contributed 
more  to  the  outcomes  than  those  of  the  team  leader. 

The  team  leader's  contrast  sensitivity  made  a  postitive 
contribution  to  FW  performance,  and  his  dark  focus  made  a 
positive  contribution  to  RW  performance.  The  flexibility  of  the 
leader's  visual  system  (NP,  FP ,  FR)  negatively  affected  FW  and  RW 
performance  while  the  gunner's  visual  flexibility  positively 
affected  FW  and  RW  performance.  Further,  the  gunner's  acuity, 
contrast  sensitivity,  and  dark  focus  played  a  role  in  RW 
performance. 

These  results  are  consistent  with  past  RADES  tests 
(SAIC,  1985)  as  well  as  other  human  factors  studies.  Visual, 
acuity  has  emerged  frequently  as  a  predictor  of  aircraft  target 
detection  and  recognition  performance  (Johnston,  1967;  Monaco  & 
Hamilton,  1984),  as  has  contrast  sensitivity  (Ginsburg  & 

Easterly,  1983;  Ginsburg,  1984).  The  dark  focus  of  accommodation 
has  been  a  prominent  factor  in  visual  task  performance  (Owens, 
1984;  Roscoe,  1985;  Norman  &  Ehrlich,  1986).  Further,  the 
flexibility  of  the  visual  system  has  been  shown  on  numerous 
occasions  to  predict  visual  task  performance  (Roscoe,  1979; 

Norman  &  Ehrlich,  1986).  Therefore,  it  is  reassuring  to  find 
that  these  vision  variables  were  consistently  correlated  with 
performance  in  RADES.  These  relationships  are  supportive  of 
earlier  RADES  findings  as  well  (Barber,  1987). 
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Table  18 

Spearman  Correlations  Between  Team  Attributes 
and  Fixed-Wing  Ranges  and  Single,  Rotary-Wing 
Times  (p<.10) 


Team  Leader 


FW 

EVENT 

AGE 

NA 

CS 

DF 

NP 

FP 

FR 

RDET 

RID 

RENG 

-.58 

-.69 

-.77 

-.56 

-.51 

.85 

.61 

-.57 

RW 

EVENT 

AGE 

NA 

CS 

DF 

NP 

' 

FP 

FR 

TDET 

TENG 

TTOT 

.66 

.61 

.63 

-.51 

.49 

|  Gunner  j 

1  FW 
| EVENT 

AGE 

NA 

CS 

DF 

NP 

FP 

FR 

RDET 

RLOCK 

RFIRE 

.60 

-.55 

.74 

.77 

.74 

.74 

RW 

EVENT 

AGE 

NA 

CS 

DF 

NP 

FP 

FR 

TDET 

TACQ 

TIFF 

TLOCK 

TTOT 

.67 

.74 

.52 

.44 

.65 

-.61 

.60 

-.58 

.60 

.52 

-.52 

-.57 

Conclusions 


This  study  has  shown  that  RADES  can  train  Forward  Area 
Air  Defense  soldiers.  However,  the  test  was  somewhat  limited  in 
its  scope  since  only  RW  training  improvements  were  analyzed. 
Training  effects  using  FW  aircraft  must  also  be  investigated. 
Upcoming  training  analyses  will  provide  more  definitive  training 
effectiveness  data. 

Some  general  inferences  can  be  made  from  the 
descriptive  statistics.  Of  the  total  engagement  period,  the 
detection  event  uses  up  about  half  of  the  time.  The 
identification  process  takes  about  two  thirds  of  the  engagement 
time  subsequent  to  detection,  and  the  fire  sequence  takes  the 
remaining  third  of  that  time.  Thus,  improvements  in  FAAD 
effectiveness  and  efficiency  may  be  found  in  selecting  personnel 
with  superior  vision.  Further,  providing  identification 
information  to  the  team,  either  over  the  command  net  or  via  some 
optical  or  imaging  device,  may  help.  Additionally,  adequate 
training  and  practice  in  a  field  environment  such  as  RADES  may 
serve  to  improve  teamwork  and  individual  skills.  Engagement 
training  with  both  friendly  and  hostile  aircraft  is  important. 

It  was  also  observed  in  this  test  and  numerous  prior 
tests  (Barber, 1987;  Drewfs,  et.  al . ,  in  press)  that  air  defense 
soldiers  appear  to  respond  quicker  and  are  more  correct  in 
identifying  hostiles  than  friends.  This  was  attributed  to  the 
tendency  for  teams  (especially  novices)  to  adopt  a  hostile  set, 
and  thus  have  a  proclivity  to  fire  when  in  doubt. 

The  differences  exhibited  in  this  study  between  single, 
sequential  and  simultaneous  rotary-wing  target  engagement 
efficiency  and  effectiveness  are  also  noteworthy.  The  presence 
of  subsequent  and  multiple  RW  aircraft  seemed  to  reduce  reaction 
time,  possibly  because  of  an  increase  in  arousal  level  of  the 
soldiers.  This  often  resulted  in  hasty  identifications  that  were 
incorrect.  The  workload  level  would  appear  to  drive  the  speed  of 
team  responses. 

ARI  investigators  also  noted  that  during  feedback 
sessions  in  this  study,  soldiers  related  that  certain  detailed 
features  they  were  trained  to  look  for  in  visual  aircraft 
identifications  were  absent  from  the  targets.  Soldiers  stated 
that  this  caused  them  to  be  initially  misled,  and  to  engage 
friendly  aircraft  such  as  the  A-10  and  CH-3.  In  addition,  they 
stated  that  this  caused  them  to  raisidentify  certain  hostile 
targets  as  friendly.  It  is  recommended,  therefore,  that  such 
features  as  electronic  countermeasures  pods,  covers,  and  parts 
not  be  taught  as  critical  features  for  visual  aircraft 
identification.  Aircraft  may  not  always  employ  these  features. 
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